Arc Institute Launches Virtual Cell Challenge to Advance AI in Healthcare

The Arc Institute, a leading non-profit research organization, has announced the launch of the Virtual Cell Challenge, a public competition designed to accelerate the development of AI models in healthcare. The challenge, sponsored by Nvidia, 10x Genomics, and Ultima Genomics, offers a grand prize of $100,000 for the most accurate machine learning model that predicts how cells will respond to genetic perturbations.

Yusuf Roohani, PhD, the machine learning group lead at the Arc Institute, is spearheading the effort. Roohani and his team are training AI models with transcriptome data to predict changes in gene expression patterns, which could help researchers develop new drugs with fewer off-target effects. "When you look at cells, they are living dynamic systems," Roohani explained. "Cells are constantly in flux, and they are dependent on the experiment."

Virtual cell models must account for biological complexity, including cell type, genetic background, and context. However, many existing single-cell datasets are impacted by technical noise, which can diminish model performance. Without standardized benchmarks and purpose-built datasets, the field has struggled to evaluate whether virtual cell models are capturing generalizable biological insights or dataset-specific patterns.

The Virtual Cell Challenge aims to address these issues by providing a standardized benchmark and dataset. The competition will evaluate models on three key metrics: performance in predicting differentially expressed genes, performance in discriminating between different perturbation effects, and general error in terms of deviation from expression counts. The interim performance of competitor models will be shared on a live leaderboard during the middle phase of the competition.

The Arc Institute's initiative follows the success of the Critical Assessment of protein Structure Prediction (CASP) competition, which has transformed protein structure prediction over 25 years. Patrick Hsu, PhD, co-founder and core investigator at Arc, emphasized the potential impact of the Virtual Cell Challenge. "We believe Arc can use the same approach to accelerate progress toward comprehensive virtual cells that could fundamentally change how we study biology and identify targets to better treat complex diseases," Hsu stated.

Emma Lundberg, PhD, associate professor at Stanford University and co-director of the Human Protein Atlas, concurs that establishing benchmarks is crucial for evaluating and comparing virtual cell models. "I expect that Arc's challenge will help align the community and accelerate the work toward performant and useful virtual cell models. Hopefully, it's the first of many standardized challenges in this space," Lundberg told GEN Edge.

Theofanis Karaletsos, senior director of AI at Chan Zuckerberg Initiative (CZI), is an active developer of virtual cell models. "At CZI, we're focused on building cutting-edge models and providing standardized evaluation frameworks to deepen the scientific community's understanding of cells. Community benchmarks are important, and we believe open competitions like Arc's are a powerful mechanism to accelerate innovation and collective progress," Karaletsos explained.

The Arc Institute was founded in 2021 by Hsu and Silvana Konermann, PhD, assistant professor of biochemistry at Stanford University and Arc's current executive director. The institute has made significant strides in data-driven AI, including the release of Evo 2, the largest publicly available AI model for biology, in collaboration with Nvidia earlier this year.

As a key challenge for AI models is making predictions outside of the training data, the Arc competition will evaluate how well competing virtual cells can predict changes in gene activity when generalizing to a new cell context. For the inaugural competition, Arc has generated a new single-cell transcriptomics dataset of 300,000 H1 human embryonic stem cells (H1 hESCs) with 300 genetic perturbations. This dataset will be deployed throughout the competition in segments for fine-tuning, validation, and testing.

Models will be evaluated on three metrics: performance in predicting differentially expressed genes, performance in discriminating between different perturbation effects, and general error in terms of deviation from expression counts. The interim performance of competitor models will be shared on a live leaderboard during the middle phase of the competition. The three teams with the top models will receive prizes valued at $100,000, $50,000, and $25,000, combining cash awards and NVIDIA DGX Cloud credits.

Registration for the competition is now open. Individual contributors as well as teams from academic institutions, biotechnology companies, and independent research organizations are eligible to participate. Final rankings will be determined solely by model performance on the final test set, which will be released in late October, one week prior to the final submission deadline. Winners will be announced in December.

As a baseline, Virtual Cell Challenge competitors will initially go head-to-head with Arc's first virtual cell model, STATE, which is designed to predict how various stem cells, cancer cells, and immune cells respond to drugs, cytokines, or genetic perturbations. STATE improved discrimination of perturbation effects on multiple large datasets by over 50% and identified differentially expressed genes across genetic, signaling, and cellular contexts.